Problem

The GlobalEndpointManager background refresh timer silently stops in multi-writer accounts, preventing the SDK from detecting topology changes (e.g., multi-write to single-write transitions).

Root Cause

In refreshLocationPrivateAsync(), when LocationCache.shouldRefreshEndpoints() returns false, the timer is never restarted:

} else {
    logger.debug("shouldRefreshEndpoints: false, nothing to do.");
    this.isRefreshing.set(false);
    return Mono.empty(); // timer dies here
}

For multi-writer accounts, shouldRefreshEndpoints() returns false when the preferred write endpoint matches the current primary -- a steady-state condition. Once that happens, no further background refreshes occur for the lifetime of the client. Bug has existed since PR #6139 (Nov 2019, point #4 in description).

Behavioral Difference with .NET SDK

The .NET SDK handles this correctly in StartLocationBackgroundRefreshLoop() -- it only terminates when canRefreshInBackground is explicitly false, continuing even when ShouldRefreshEndpoints() returns false.

Fix

Add startRefreshLocationTimerAsync() to the else branch of refreshLocationPrivateAsync():

} else {
    logger.debug("shouldRefreshEndpoints: false, nothing to do.");
    if (!this.refreshInBackground.get()) {
        this.startRefreshLocationTimerAsync();
    }
    this.isRefreshing.set(false);
    return Mono.empty();
}

Unit Tests

6/6 pass:

• backgroundRefreshForMultiMaster: Updated assertion -- timer must keep running
• backgroundRefreshDetectsTopologyChangeForMultiMaster: New -- simulates MW-to-SW transition via mock

Live DR Drill Validation (4 Scenarios)

Date: 2026-04-10 22:10Z -- 2026-04-11 00:32Z | Branch: fix/background-refresh-multi-writer @ 2048abeca

All scenarios used Direct + Gateway modes simultaneously. Kusto data from BackendEndRequest5M (Direct) and Request5M (Gateway).

Accounts

Scenario 1: MW -- Offline Secondary Region

Global endpoint, preferred = West US. Offline West US, observe failover to East US.

PASS -- Failover to East US in ~4 min. 32 GEM refreshes. West US traffic resumed after restore.

Scenario 2: MW -- MW-to-SW-to-MW Transition (Core PR validation)

Regional endpoint (westus.documents.azure.com), no preferred region. Disable then re-enable multi-write.

PASS -- Both transitions detected. MW-to-SW in ~3.5 min (writes shifted to EUS). SW-to-MW in ~1 min (writes returned to WUS). 28 GEM refreshes.

Scenario 3: SW -- Switch Write Region

Global endpoint, preferred = East US. Switch write EUS-to-WUS.

PASS -- Writes on WUS within 1 Kusto bucket. 20 GEM refreshes.

Scenario 4: SW -- Offline Write Region

Global endpoint, preferred = East US. Offline East US.

PASS -- Full failover to WUS in ~3 min. 32 GEM refreshes.

Backend Success Rates

Direct mode (BackendEndRequest5M)

Gateway mode (Request5M)

All errors (403/3 write-to-read-only, 404/1002 session-not-available) were auto-retried by the SDK -- zero user-visible failures.

Verdict

// Direct mode ops (BackendEndRequest5M)
BackendEndRequest5M
| where TIMESTAMP between (datetime({start}) .. datetime({end}))
| where GlobalDatabaseAccountName == '{account}'
| where UserAgent has 'dr-' | where ResourceType == 2
| extend Workload = extract('(dr-[a-z-]+-[a-z]+)', 1, UserAgent)
| summarize Requests=sum(SampleCount) by bin(TIMESTAMP, 5m), Region, Workload

// Gateway mode ops (Request5M -- lowercase columns)
Request5M
| where TIMESTAMP between (datetime({start}) .. datetime({end}))
| where globalDatabaseAccountName == '{account}'
| where userAgent has 'gw'
| extend Workload = extract('(dr-[a-z-]+-[a-z]+)', 1, userAgent)
| summarize Total=sum(SampleCount), Success=sumif(SampleCount, statusCode < 400) by Workload
| extend SuccessRate=round(100.0 * Success / Total, 3)

// Write region transitions (MgmtDatabaseAccountTrace)
MgmtDatabaseAccountTrace
| where TIMESTAMP between (datetime({start}) .. datetime({end}))
| where GlobalDatabaseAccount == '{account}'
| project TIMESTAMP, Location, LocationType, FederationId, Status

Changes

• 1 file changed, 10 insertions (GlobalEndpointManager.java)
• 1 file changed, 50 insertions, 1 deletion (GlobalEndPointManagerTest.java)